Sampling Techniques for the Nystrom Method
نویسندگان
چکیده
The Nyström method is an efficient technique to generate low-rank matrix approximations and is used in several large-scale learning applications. A key aspect of this method is the distribution according to which columns are sampled from the original matrix. In this work, we present an analysis of different sampling techniques for the Nyström method. Our analysis includes both empirical and theoretical components. We first present novel experiments with several real world datasets, comparing the performance of the Nyström method when used with uniform versus non-uniform sampling distributions. Our results suggest that uniform sampling without replacement, in addition to being more efficient both in time and space, produces more effective approximations. This motivates the theoretical part of our analysis which gives the first performance bounds for the Nyström method precisely when used with uniform sampling without replacement.
منابع مشابه
Recursive Sampling for the Nystrom Method
We give the first algorithm for kernel Nyström approximation that runs in linear time in the number of training points and is provably accurate for all kernel matrices, without dependence on regularity or incoherence conditions. The algorithm projects the kernel onto a set of s landmark points sampled by their ridge leverage scores, requiring just O(ns) kernel evaluations and O(ns) additional r...
متن کاملLess is more: optimal learning with subsampling regularization∗
In this talk, we discuss recent results on common techniques for scaling up nonparametric methods such as kernel methods and Gaussian processes. In particular, we focus on data dependent and independent sub-sampling methods, namely Nystrom and random features, and study their generalization properties within a statistical learning theory framework. On the one hand we show that these methods can...
متن کاملNystrom Method for Approximating the GMM Kernel
The GMM (generalized min-max) kernel was recently proposed [5] as a measure of data similarity and was demonstrated effective in machine learning tasks. In order to use the GMM kernel for large-scale datasets, the prior work resorted to the (generalized) consistent weighted sampling (GCWS) to convert the GMM kernel to linear kernel. We call this approach as “GMM-GCWS”. In the machine learning l...
متن کاملEfficient Algorithms and Error Analysis for the Modified Nystrom Method
Lemma 8. Given an m × m symmetric matrix A and a target rank k, we let C1 contain the c1 columns of A selected by a column sampling algorithm such that the following inequality holds: ∥∥A− PC1A∥∥2F ≤ f∥∥A−Ak∥∥2F . Then we select c2 = kf −1 columns to construct C2 and c3 = (c1+ c2) −1 columns to construct C3, both using the adaptive sampling according to the residual B1 = A − PC1A and B2 = A − P...
متن کاملNystrom plus correction method for solving bound-state equations in momentum space.
A method is presented for solving the momentum-space Schrödinger equation with a linear potential. The Lande-subtracted momentum-space integral equation can be transformed into a matrix equation by the Nystrom method. The method produces only approximate eigenvalues in the cases of singular potentials such as the linear potential. The eigenvalues generated by the Nystrom method can be improved ...
متن کامل